Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(endpoints): scrape endpoints and not services #148

Merged
merged 8 commits into from
Apr 23, 2021

Conversation

paologallinaharbur
Copy link
Member

@paologallinaharbur paologallinaharbur commented Mar 9, 2021

Fix #27

The idea is that when a service is detected the underlying endpoints are fetched. For backward compatibility the metric directly coming from the service is kept
Screen Shot 2021-03-10 at 11 57 09 AM

Two new config options have been added ScrapeServices (default true) and ScrapeEndpoints(default false)

After merging we should update documentation and the helm chart to ask for listing/watching permission on endpoints

@roobre roobre added bug Categorizes issue or PR as related to a bug. and removed bug Categorizes issue or PR as related to a bug. labels Mar 10, 2021
@paologallinaharbur paologallinaharbur changed the title [WIP] fix(endpoints): scrape endpoints and not services fix(endpoints): scrape endpoints and not services Mar 11, 2021
@paologallinaharbur paologallinaharbur requested a review from a team March 11, 2021 11:38
@paologallinaharbur paologallinaharbur force-pushed the fix/ScrapeEndpoints branch 2 times, most recently from c948c40 to 9a8bc48 Compare March 11, 2021 12:00
@paologallinaharbur
Copy link
Member Author

Currently after merging this we will be sending more data than before.

We should decide to filter out some targets, es:

func (k *KubernetesTargetRetriever) GetTargets() ([]Target, error) {
	length := 0
	k.targets.Range(func(_, _ interface{}) bool {
		length++
		return true
	})
	targets := make([]Target, 0, length)
	k.targets.Range(func(_, y interface{}) bool {
		for _, t := range y.([]Target) {
			if t.Object.Kind == "service" && excludeServiceTargets {
				continue
			}
			if t.Object.Kind == "endpoints" && excludeEndpointsTargets {
				continue
			}
			targets = append(targets, t)
		}
		return true
	})

	return targets, nil
}

Copy link
Contributor

@gsanchezgavier gsanchezgavier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to have these behind a feature flag keeping the old behavior by default. Or have a mayor release. I would also propose to have this feature in beta for some time.

Could you try the load test to se how is the result with the new feature?

internal/pkg/endpoints/kubernetes.go Show resolved Hide resolved
@paologallinaharbur
Copy link
Member Author

@gsanchezgavier I added two feature flags and given what we talked about offline in the past days, for now the option to scrape endpoints will be disabled by default.

I'll update the documentation and the HelmChart to further clarify this

Copy link
Contributor

@gsanchezgavier gsanchezgavier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@paologallinaharbur
Copy link
Member Author

After merging and releasing we need to update documentation and expose the new config through the helm charts

@paologallinaharbur paologallinaharbur merged commit 4d4d75d into main Apr 23, 2021
@paologallinaharbur paologallinaharbur deleted the fix/ScrapeEndpoints branch April 23, 2021 08:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Services are scraped directly instead of service endpoints
4 participants